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FOREWORD 


The  US  Army  Research  Institute  (ARI) ,  in  cooperation  with 
TRADOC  and  its  schools,  performs  research  and  development  on  ways 
to  achieve  more  cost-effective  training.  In  1987,  ARI  joined  with 
TRADOC  and  the  US  Army  Ordnance  Center  and  School  (USAOC&S)  in  a 
partnership  at  Aberdeen  Proving  Ground  (APG)  ,  Maryland,  to 
identify  and  solve  maintenance  training  problems.  The  partnership 
was  defined  by  a  memorandum  of  understanding  (MOU)  entitled 
"Establishment  of  a  Training  Technology  Field  Activity  (TTFA)  at 
the  USAOC&S,"  and  dated  10  May  1987. 

The  present  report  is  one  result  of  that  partnership.  The 
work  was  carried  out  as  part  of  Task  344  by  members  of  the 
Logistics  Training  Technologies  Technical  Area  of  the  Training 
Research  Laboratory  in  order  to  provide  practical  guidelines  for 
designing  maintenance  training  which  will  result  in  a  high  degree 
of  skill  retention  and  transfer. 

This  and  other  products  of  the  TTFA  were  briefed  to  the 
Deputy  Assistant  Commandant,  USAOC&S,  in  October,  1988.  It  will 
be  used  to  help  develop  model  training  instruction  and  plan 
additional  TTFA  projects  at  USAOC&S.  Its  value  was  recognized  by 
MG  Ball,  Commandant,  USAOC&S,  in  a  letter  of  appreciation  to  the 
author.  In  October  1989  this  project  was  briefed  in  detail  to  a 
representative  of  the  Israeli  Army,  who/7 showed  great  interest  in 


Technical  Director 
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APPLICATION  OF  ARI  SKILL  RETENTION  MODEL  TO  WHEEL  VEHICLE 
MAINTENANCE  TASKS 


EXECUTIVE  SUMMARY 


Requirement: 

Support  a  Training  and  Doctrine  Command  (TRADOC)  effort  to 
identify  performance  deficiencies  among  63W  mechanics  in  FORSCOM 
units.  Recommend  corrective  training  procedures. 


Procedure: 

In  order  to  identify  potential  performance  deficiencies,  we 
administered  the  Skill  Retention  Model  to  seven  subject  matter 
experts  (SMEs)  at  the  U.S.  Army  Ordnance  Center  &  School 
(USAOC&S) ,  Aberdeen  Proving  Ground  (APG) .  The  SMEs  rated  nine 
critical  tasks  on  ten  characteristics.  From  these  ratings  and  a 
table  in  the  Model,  we  generated  skill  decay  curves  for  each 
task.  We  also  identified  reasons  for  the  decay.  Then  we 
inferred  and  recommended  corrective  training  procedures  from  the 
10  subscales  of  the  Model. 


Findings: 

(1)  It  is  feasible  and  practicable  to  administer  the  Skill 
Retention  Model  to  SMEs  at  APG.  A  SME  can  easily  evaluate  two 
tasks  per  hour. 

(2)  Three  of  the  tasks  rated  are  subject  to  severe  skill 
decay:  replace  and  time  the  fuel  injector  pump;  diagnose  engine 
that  starts  but  stalls;  diagnose  engine  that  cranks  but  doesn't 
start.  The  SMEs  disagreed  among  themselves  on  the  task  charac¬ 
teristics  and,  therefore,  on  the  retention  for  remove/ replace  the 
steering  gear.  The  SMEs  agreed  that  the  remaining  five  tasks 
were  resistant  to  skill  decay. 

(3)  Out  of  ten  task  characteristics,  five  account  for  the 
results  above,  i.e.,  quality  of  technical  manuals,  number  of 
facts  to  memorize,  difficulty  of  facts  to  recall,  mental  require¬ 
ments,  and  motor  control  requirements.  The  SMEs  agreed  that  the 
tasks  differed  on  these  task  characteristics  and  did  not  agree 
that  the  tasks  differed  on  the  other  five  characteristics. 

(4)  From  the  10  characteristics,  we  inferred  and  recom¬ 
mended  ways  to  reduce  skill  decay  through  improved  training  and 
job  aiding.  We  also  estimated  the  increase  in  retention  time  for 
each  training  solution. 
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(5)  We  concluded  that  the  model  could  be  used  routinely  to 
help  prioritize  tasks  for  training  and  to  evaluate  cost- 
effectiveness  tradeoffs. 


Utilization  of  Findings: 

(1)  Help  USAOC&S  and  TRADOC  decide  whether  to  use  the 
Retention  Model  routinely  for  training  development  and  cost- 
effectiveness  analysis. 

(2)  Help  the  Training  Technology  Field  Activity  produce  and 
test  training  development  guidelines  for  programs  of  instruction 
at  USAOC&S . 
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APPLICATION  OF  ARI  SKILL  RETENTION  MODEL  TO  WHEEL 
VEHICLE  MAINTENANCE  TASKS 


1.0  INTRODUCTION 


1.1  Overview 

This  report  is  one  of  a  series  in  the  Training  Technology 
Field  Activity  (TTFA)  program  to  improve  automotive  maintenance 
training  at  the  U.S.  Army  Ordnance  Center  &  School  (USAOCS) 
(Ramsay,  Kessler,  &  Mirabella,  in  preparation;  Mirabella,  Mac- 
pherson,  &  Patterson,  1988;  &  Kessler,  in  preparation).  It  shows 
how  we  used  the  ARI  skill  retention  model  to  identify  critical 
63W10  tasks  which  are  subject  to  severe  forgetting  (Rose,  Manning, 
Radtke,  &  Ford,  1984;  Rose  et  al.,  1985a;  Rose,  Radtke,  Shettel,  & 
Hagman,  1985b) .  The  report  supports  a  larger  TTFA  effort  to  iden¬ 
tify  maintenance  skill  deficiencies  (Applied  Sciences  Associates, 
1988)  .  It  also  demonstrates  how  the  model  can  be  used  by  school 
SMEs  and  quantifies  the  effort  required. 

1 . 2  Background 

a.  The  Army  Research  Institute  has  joined  USAOCS  and  the 
Training  Development  &  Analysis  Directorate  of  TRADOC  in  a  TTFA 
at  Aberdeen,  Maryland.  The  goal  of  the  TTFA  is  to  identify  short¬ 
falls  in  maintenance  performance  and  then  build  one  or  more  model 
course  segments  to  demonstrate  how  those  shortfalls  can  be 
reduced  through  improved  training  methodology.  To  identify  the 
shortfalls,  the  TTFA  representatives  will  observe  63Wl0s  at  work 
in  FORSCOM  active  and  reserve  component  maintenance  units  such  as 
those  at  Forts  Riley  and  Stewart. 

b.  The  results  of  the  present  study  will  supplement  the  field 
evaluations  and  could  help  us  interpret  the  data  from  those 
evaluations.  For  example  if  we  are  unable  to  observe  some 
critical  tasks  in  the  field  we  could  substitute  data  obtained  via 
the  retention  model.  Or,  if  we  find  that  Task  X  is  done  well  at 
Riley  but  poorly  at  Stewart,  we  could  turn  to  the  skill  retention 
data  to  find  that  this  task  is  not  well  remembered  unless  it  is 
practiced  often.  The  "discrepancy”  in  field  data  would  then  make 
some  sense,  if  we  also  found  that  Task  X  is  performed  more 
frequently  at  Fort  Riley. 

c.  To  do  the  analysis,  we  selected  the  1985  revision  of  the 
"User's  Manual  for  Predicting  Military  Task  Retention"  (Rose  et 
al.,  1985b).  The  skill  retention  model  is  based  upon  extensive 
research  into  the  task  characteristics  that  are  associated  with 
skill  retention.  This  version  of  the  skill  retention  model  was 
developed  using  78  Army  tasks  and  military  personnel  (Hagman  et 
al.,  1986).  Furthermore,  it  has  extremely  high  validity  and 
reliability  (Rose  et  al.,  1984,  1985a;  Hagman,  1986).  It, 
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therefore,  can  be  used  with  reasonable  confidence  for  TTFA 
purposes. 

d.  The  model  is  based  upon  ten  task  characteristics  that 
were  found  to  influence  skill  retention  such  as  the  quality  of 
the  available  job  aids,  the  number  of  steps  in  the  task,  and 
several  mental  characteristics  of  task  performance.  The  "User's 
Manual"  presents  these  characteristics  as  ten  scales.  Each  scale 
is  divided  into  a  small  number  of  categories.  The  subject  matter 
expert  (SME)  examines  the  task  and  determines,  for  each  scale, 
the  category  that  best  describes  the  task.  Next  the  SME  sums 
these  retention  values  to  determine  the  total  retention  score  for 
the  task.  Finally  the  user  refers  to  a  table  which  translates  the 
retention  score  intc  the  percent  of  soldiers  who  will  be  able  to 
perform  the  task  without  training  for  any  period  from  a  week  up 
to  a  year.  With  the  "User's  Manual"  the  user  can  evaluate  tasks 
which  are  so  easy  that  they  can  be  remembered  without  practice 
for  a  year  as  well  as  tasks  so  difficult  that  only  a  few  per¬ 
formers  can  do  them  properly  after  a  week  off. 

1.3  Objectives  of  the  Study 

a.  The  principal  objective  was  to  provide  explanatory 
information  on  task  performance  deficiencies  in  FORSCOM  units  by 
generating  theoretical  curves  of  pure  forgetting  without  inter¬ 
vening  practice  for  the  unit  tasks  and  based  on  task  character¬ 
istics  known  to  be  associated  with  forgetting.  This  report  is 
complete  in  the  sense  that  it  provides  all  the  information  that 
the  reader  needs  to  apply  the  methodology. 

b.  A  secondary  goal  was  to  support  a  related  TTFA  project 
"State  of  the  Art  Analysis  of  Research  on  Skill  Acquisition, 
Retention,  and  Transfer."  We  want  to  use  the  results  of  that 
analysis  to  design  improved  training  for  a  testbed  maintenance 
task  that  is  subject  to  severe  skill  decay.  Results  of  the 
current  study  will  help  us  select  an  appropriate  task. 

c.  A  third  goal  was  to  find  out  if  the  retention  model  could 
be  used  easily  at  Aberdeen.  The  skill  decay  model  is  an  inexpen¬ 
sive,  quick,  and  valid  way  to  prioritize  tasks  for  training 
emphasis.  We  need  to  ensure,  however,  that  the  method  does  not 
impose  an  excessive  personnel  burden. 
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2.0  METHOD 


2.1  Subjects 

The  raters  were  seven  instructors  from  the  Ordnance  School 1 s 
Wheel  Vehicle  Department  at  Edgewood  MD.  Each  instructor  had 
served  in  an  Army  maintenance  unit.  During  the  interview  each 
demonstrated  that  he  knew  each  of  the  tasks  that  he  rated  by  such 
behaviors  as  discussing  the  strengths  and  weaknesses  of  the 
appropriate  technical  manuals  from  memory.  No  rater  could  be 
considered  senior  to  the  others,  although  the  raters  in  the 
second  group  regarded  one  of  their  number  as  more  knowledgeable 
concerning  removal  and  replacement  of  the  steering  gear.  One 
rater  participated  in  both  rating  sessions. 

2.2  Research  Instruments.  Three  sets  of  instruments  were  used: 

a.  Procedure  lists  fc_*  nine  tasks.  The  task  lists 
(Appendices  Cl  -C9)  were  prepared  by  Applied  Science  Associates 
(1988)  from  Army  technical  manuals,  in  order  to  evaluate  63W 
maintenance  performance  in  FORSCOM  active  and  reserve  components. 
The  first  five  tasks  in  the  list  below  were  selected  as  important 
by  school  SMEs  and  school  training  development  personnel.  Tasks  5 
-  9  were  found  to  be  frequently  performed  in  the  field  by  Applied 
Science  Associates.  Thus  task  5  was  considered  important  and  was 
performed  frequently  in  the  field.  Each  list  included  the  task 
steps,  an  indication  of  which  steps  had  to  be  performed  sequen¬ 
tially,  the  end  products  of  the  sub-tasks,  the  safety  require¬ 
ments,  as  well  as  the  tools  and  materials  needed.  The  tasks 
were: 

(1)  Diagnose  hard  starting 

(2)  Diagnose  loss  of  engine  power 

(3)  Diagnose  stalling  engine 

(4)  Diagnose  engine  which  cranks  but  doesn't  start 

(5)  Replace  fuel  injector  pump 

(6)  Remove  and  replace  the  steering  gear  on  an  M35A2 

(7)  Remove  and  replace  the  axle  on  an  M813A1 

(8)  Remove  and  replace  the  clutch  on  an  M35A2 

(9)  Remove  and  replace  the  transmission  on  a  M35A2. 
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b.  The  Qur stionnaire  for  Predicting  Military  Task  Retention 
(Rose  et  al.,  1985b).  The  Questionnaire  requires  that  the  user 
rate  the  task  on  10  scales  that  have  been  demonstrated  to  predict 
how  quickly  soldiers  will  forget  tasks.  The  scales  are:  1) 
presence  of  jcb/memory  aid,  2)  job/memory  aid  quality,  3)number 
of  steps  in  the  task,  4)  sequence  of  steps,  5)  feedback,  6)  time 
pressure,  7)  mental  requirements,  8)  number  of  facts  to  remember, 
9)  difficulty  of  remembering  the  facts,  10)  motor  control 
requirements.  (See  Appendix  B  for  definitions). 

c.  Tables  for  estimating  skill  retention.  Rose  et  al., 
(1985b)  and  Hagman  (1986)  provide  tables  which  allow  the  user  to 
estimate  the  percentage  of  soldiers  who  will  be  able  to  perform 
a  task  after  periods  of  no  practice  ranging  from  one  week  to  1 
year.  These  tables  are  provided  in  Appendix  B. 

2 . 3  Procedure 

a.  We  administered  the  questionnai  e  to  two  groups  of  four 
SMEs.  As  stated,  one  SME  was  in  both  groups.  The  first  rating 
session  concentrated  on  the  tasks  that  had  been  recommended  by 
the  Ordnance  School.  The  second  session  was  devoted  to  tasks 
which  Applied  Science  Associates  had  observed  were  being 
performed  in  the  field.  Task  5,  "remove  and  replace  the  fuel 
injector  pump",  was  rated  by  both  groups  of  SMEs. 

b.  We  interviewed  each  rater  independently.  We  gave  the 
rater  a  questionnaire  and  the  "User's  Manual”  and  then  "talked” 
him  through  the  ratings.  The  task  analyses  and  the  questionnaire 
for  predicting  military  task  retention  were  amplified  beyond  the 
manual,  according  to  the  protocol  in  Appendix  C.  In  addition  we 
also  "highlighted"  the  most  relevant  sections  of  the  User's 
Manual  so  that  the  raters  could  find  the  information  defining  the 
scale  values  more  quickly.  At  the  request  of  our  SMEs,  we  made 
two  additions  to  the  protocol .  In  response  to  our  experts ' 
uneasiness  at  rating  recent  graduates  we  defined  our  mechanic  as 
one  with  two  years  experience.  We  standardized  the  location  of 
the  operations  by  specifying  that  all  maintenance  took  place  in 
the  shop  rather  than  in  the  field  on  maneuvers. 

c.  After  the  SMEs  finished  their  ratings,  we  assembled  them 
into  a  group.  The  group  then  discussed  each  rating  of  each  task 
until  it  reached  a  consensus  or  seemed  unable  to  do  so.  If  it 
failed  to  agree,  we  used  the  modal  value,  as  required  by  the 
manual.  Three  situations  occurred  where  we  decided  to  present 
the  disagreement  among  the  observers.  We  present  the  two  sets  of 
data  that  were  produced  for  these  three  tasks.  We  describe  the 
situations  producing  the  two  sets  of  data  below: 

(1)  In  the  "engine  cranks,  doesn't  start  diagnosis"  the  four 
raters  split  into  two  equal  groups  and  were  unwilling  to 
shift.  The  manual  states  that  both  sets  "jf  results  should  be 
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used. 


(2)  In  the  "remove/replace  steering  gear"  task  the 
rater  considered  most  expert  by  the  others  scored  the 
task  on  the  basis  that  the  job  aids  (manuals)  would  not 
be  available  on  the  shop  floor,  then  was  unable  to 
attend  the  group  session.  We  include  his  data  because 
he  was  considered  most  knowledgeable,  his  responses 
demonstrate  the  predicted  effect  of  lack  of  technical 
manuals,  and  they  correspond  with  the  informal 
observations  of  an  Ordnance  School  field  observer. 
Furthermore,  they  agreed  with  those  of  Schruman,  Joyce 
and  Porsche  (1980)  who,  in  their  study  of  Army  wheeled 
vehicle  mair  enance,  reported  that  TMs  were  unavailable, 
obsolete,  and  that  "mechanics  who  did  try  to  use  the 
manuals  seemed  unfamiliar  with  the  layout  and  indexing" 
of  them. 

(3)  The  "remove/replace  fuel  injector  pump"  was  rated 
in  both  sessions  because  it  was  selected  as  important  by 
school  personnel  and  performed  frequently  in  the  field. 
Thus  it  was  rated  by  the  group  of  raters  who  rated  the 
important  tasks  and  the  group  who  rated  the  frequently 
performed  tasks. 
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3 . 0  RESULTS 


3.1  Application  of  the  Skill  Retention  Model. 

a.  Total  labor  for  application  of  the  model  to  8  SMEs  was 
26  hours  or  2.8  hours  per  task.  Total  clock  time  was  10  hours  or 
1.0  hour  per  task.  Each  rater,  working  individually  with  an 
interviewer,  took  an  hour  to  rate  the  5  tasks,  i.e.  12  minutes 
per  task,  per  rater.  The  group  discussion,  led  by  the  inter¬ 


viewer,  took 

another 

hour 

for 

each  group, 

i.e. 

12  minutes 

per 

task.  Interviewer  labor 

was 

12  min 

per 

task, 

per 

rater 

for 

independent 

sessions, 

and 

12 

minutes 

per 

task 

for 

the  group 

sessions. 

b.  The 

equations 

below  were  used 

to 

calculate 

the  total 

effort  required: 

(1)  SME  labor  (individual)  =  12  min  x  number  of  tasks  x 
number  of  raters 

(2)  SME  labor  (group  session)  =  12  min  x  number  of  tasks  x 
number  of  raters 


(3)  Interviewer  (individual)  =  12  min  x  number  of  tasks  x 
number  of  raters 

(4)  Interviewer  (group  session)  =  12  min  x  number  of  tasks 
3.2  Data  on  Skill  Retention 

a.  The  retention  data  appear  in  Table  l,  which  is  a  task 
scoring  sheet.  The  data's  implications  for  task  performance  are 
presented  graphically  in  Figures  1  and  2.  The  table  gives  the 
group  ratings  for  each  task  and  questionnaire  item.  The  tasks 
are  presented  in  the  left  column  in  two  groups  —  "Diagnostic” 
and  "Remove  &  Replace."  Total  task  ratings  are  shown  under 
"Total  Score."  The  numbered  columns  identify  the  scales.  The 
labels  for  the  columns  present  the  scale  values  and  instructions 
for  skipping  certain  scales.  The  scale  value  assigned  by  the 
group  to  the  task  is  on  the  row  for  that  task.  The  cell  for  a 
column  is  filled  with  a  dash  when  the  instructions  require  that 
the  scale  be  skipped.  The  right-most  column  translates  the  total 
scores  into  the  time  lapse  since  last  proficient  performance  that 
is  required  for  50%  of  the  troops  to  forget  the  task.  Thus  for 
the  task  "engine  starts,  but  stalls"  half  the  mechanics  would  be 
unable  to  perform  the  task  to  standards  if  they  had  not  performed 
the  task  for  about  6.5  weeks. 
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b.  Note  that  group  ratings  varied  by  one  scale  step  or  less 
for  scales  l,  3,  4,  and  5.  Technical  manuals  exist  for  all  the 
tasks.1  All  tasks  have  more  than  10  steps.  All  tasks  are 
sequential  or  mostly  sequential.  All  tasks  were  rated  easy  to 
perform  within  the  time  allotted.  Scores  for  the  remaining 
scales,  however,  varied  considerably,  e.g.,  the  range  of  values 
for  amount  of  feedback  is  0  to  22.  Scales  that  differentiated 
among  tasks  are  listed  below  in  order  of  decreasing  effect  on 
retention. 

(1)  technical  manual  quality, 

(2)  mental  requirements, 

(3)  feedback  provided  by  the  task, 

(4)  number  of  facts  to  memorize, 

(5)  ease  of  recalling  the  memorized  facts,  and 

(6)  and  physical  skill  required. 

We  identified  and  rank  ordered  the  preceding  scales  by 
examining  the  columns  of  Table  1.  We  observed  that  the  tasks 
differed  by  large  numerical  values  on  the  scales  listed  above,  and 
that  these  values  represented  adjacent  scale  categories. 
Examination  of  the  Performance  Predication  Tables  in  Rose  et 
al.  (1985b)  indicated  that  differences  of  these  magnitudes  would 
increase  retention  times  by  at  least  50%,  therefore  we  considered 
these  scales  to  identify  possible  areas  for  training  intervention. 

c.  The  task  retention  tables  in  the  User's  Manual  were  used 

to  create  two  graphs  presenting  12  retention  curves  for  our  nine 
tasks.2  The  results  are  presented  in  Figures  1  and  2.  Figure  1 
shows  that  skills  for  low  to  moderate  scoring  tasks  decay  very 
rapidly  compared  to  those  for  tasks  with  higher  scores.  For 

example,  at  one  month  the  passing  rate  ranges  from  90%  for  "engine 

hard  to  start"  down  to  33%  for  "engine  cranks,  not  start  2"  even 
though  the  soldiers  were  able  to  get  a  "GO"  on  all  tasks  a  month 
earlier. 

d.  Another  way  to  see  this  difference  in  decay  rates  is  to 

follow  a  horizontal  line  from,  for  instance,  the  50%  point  on  the 
Y-axis,  mark  where  that  line  crosses  each  of  the  retention  curves. 
Figure  2  shows  that,  without  practice  in  replacing  the  fuel 
injector  pump,  50%  of  repairmen  would  lose  that  replacement  skill 
in  only  2  months,  according  to  the  first  group  of  raters.  In 

contrast  it  would  be  8  months  before  50%  of  repairmen  forget  how 

to  remove  and  replace  an  axle. 

e.  Tasks  achieving  scores  above  180  are  not  forgotten  in  a 
year,  the  maximum  time  spanned  by  the  tables  in  Rose  et  al. 
(1985b).  Since  removing  and  replacing  both  the  clutch  and  the 
transmission  scored  180  or  higher,  the  curves  for  these  two  tasks 
are  plotted  as  a  single  line  across  the  top  of  Figure  2. 


Table  1 

Task  Characteristics  Rating 
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Predicted  engine  diagnostic  performanc 


Clutch  or  Transmissio 


Predicted  remove/replace  performanc 


f.  As  described  in  the  procedure  section  multiple  scores  are 
presented  for  three  tasks: 

(1)  The  task  "engine  cranks,  doesn't  start"  two  SMEs 
believed  that  the  mental  requirements  (scale  7)  best  matched 
the  "simple"  examples  in  the  manual,  whereas  the  other  two 
believed  that  the  average  63W  mechanic  found  the  mental 
requirement  more  like  the  "complex"  examples.  Although  these 
responses  were  adjacent  on  the  rating  scale,  tasks  with 
complex  mental  requirements  are  retained  only  half  as  long  as 
those  with  simple  mental  requirements. 

(2)  "Remove  and  replace  fuel  injector  pump"  received 
ratings  from  the  two  rating  groups.  The  two  ratings  differ 
primarily  because  the  first  group  believed  that  the  task 
provided  feedback  only  at  the  end,  when  the  mechanic  tested 
the  pump.  The  second  group  considered  that  each  step 
provided  feedback,  that  the  mechanic  could  tell  if  he  had 
miss-threaded  a  nut,  for  instance.  Thus  the  difference 
between  these  ratings  is  a  result  of  differing  definitions  of 
the  scope  of  feedback  to  be  considered  in  the  evaluation. 

(3)  In  "remove  and  replace  steering  gear"  the  most  expert 
SME  on  that  task  stated  that  manuals  were  not  used  on  the 
shop  floor.  The  remaining  SMEs  scored  the  manuals 
"excellent"  and  stated  that  they  were  available  to  the 
mechanics  on  the  shop  floor.  Thus  we  present  the  "without 
manual"  results  with  the  caveat  that  they  represent  the 
judgment  of  one  individual  rather  than  a  consensus.  As 
stated  in  the  procedure  section  other  observers  have 
commented  on  the  lack  of  manuals  on  the  shop  floor  in  the 
field. 
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4.0  DISCUSSION 


4.1  Overview 

a.  The  results  show  that  it  is  feasible  to  use  the  Skill 
Retention  Model  as  a  tool  for  differentiating  and  prioritizing 
critical  tasks  at  the  USAOCS.  The  model  can  be  used  to  identify 
those  tasks  most  susceptible  to  skill  decay  and  therefore  most  in 
need  of  special  training  resources.  Given  its  high  reliability 
and  validity  the  model  provides  a  very  cost-effective  supplement 
to  field  studies  since  the  current  procedure  can  process  1  task 
per  clock  hour.  A  similar  number  of  tasks,  studied  in  the  field 
is  currently  taking  about  8  months,  at  a  cost  of  about  $200,000 
in  contract  funds,  in  addition  to  extensive  labor  by  TRADOC  and 
ARI  personnel.  The  model,  however,  is  not  a  substitute  for  good 
field  evaluation,  which  reflects  many  factors  other  than  pure 
forgetting  and  which  provides  details  not  available  from  the 
model.3  However,  it  is  a  critical  supplement,  since  it  permits 
the  user  to  quickly  evaluate  large  numbers  of  tasks. 

b.  This  application  of  the  model  demonstrated  that  SMEs 
could  agree  on  appropriate  scale  values  a  large  proportion  of  the 
time,  since  they  agreed  to  disagree  on  only  one  task  of  the  nine 
studied,  "engine  cranks,  doesn't  start."  The  results  of  the  two 
administrations  of  "Remove  and  replace  fuel  injector  pump" 
suggest  that  intra-group  agreement  on  a  task  does  not  rule  out 
different  ratings  by  different  groups.  Thus,  when  multiple 
groups  are  being  used  to  rate  tasks  some  of  the  same  tasks  should 
be  rated  by  all  the  groups,  so  that  any  biases  can  be  corrected. 
The  correction  might  consist  of  immediate  retraining  of  the 
discrepant  groups  or  post-hoc  adjustment  of  their  scores,  thereby 
making  all  group  ratings  more  comparable. 

c.  The  results  also  point  to  clues  as  to  how  to  overcome  the 
performance  deficiencies  predicted  by  this  study.  They  do  so 
by  identifying  potential  task  and  training  variables  which 
account  for  low  task  ratings,  and  especially  variables  which 
differentiate  among  the  tasks  studied.  We  discuss  these  clues 
further  in  the  next  section. 


4.2  Training  Development/Job  Aiding  Implications  of  the  Results 
a.  Job  Aids. 

(1)  Problem 

(a)  A  major  difference  among  the  tasks  was  the  varying 
quality  of  available  technical  manuals  (TMs) .  TMs  don't  help 
much  if  they  are  hard  to  search  through.  If  they  were  judged 
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"excellent"  then  the  task  would  be  performed  correctly  for  twice 
as  long  as  the  same  task  with  less  than  excellent  manuals.  The 
curves  for  the  "remove  and  replace  steering"  demonstrate  how 
important  available  excellent  manuals  can  be. 

(b)  The  major  difference  among  the  TMs'  effectiveness  for 
the  different  tasks  was  in  the  number  of  manuals  required  to 
perform  the  task.  The  low  rated  tasks  tended  to  require 
searching  multiple  manuals  to  do  the  job  whereas  the  other  tasks 
did  not. 

(2)  Training  Solutions. 

(a)  Increase  the  amount  and  quality  of  training  in  how  to 
use  TMs,  especially  where  multiple  TMs  are  required. 

(b)  Place  even  more  emphesis  on  requiring  63W  students  in 
Phases  1  and  2  to  use  manuals  in  all  aspects  of  training.  Give 
much  more  experience  in  the  tasks  requiring  multiple  manuals. 

(c)  Develop  special  aiding  techniques  for  tasks  requiring 
multiple  manuals  or  very  difficult  searches  through  the  TMs. 
e.g.  special  checklists  which  pull  procedures  together  in  one 
place.  (See  Swezey  (1987)  and  Elliot  &  Joyce  (1971)  for  detailed 
guidance)  .  Use  the  results  from  applications  of  the  skill 
retention  model  to  identify  the  tasks  most  in  need  of  special  job 
aiding. 

(3)  Example  of  Pay-off.  For  the  "engine  cranks  -  doesn't 
start"  problem,  the  50%  failure  rate  point  could  move  from  4  days 
to  3.5  weeks  with  improved  job  aiding.  Excellent  job  aiding 
places  the  memory  in  the  aid  so,  essentially,  the  job  is  not 
forgotten  as  long  as  the  job  aid  is  used. 


b.  Mental  Requirements 
(1)  Problem. 

(a)  We  could  double  retention  by  doing  a  better  job  of 
teaching  students  to  analyze  maintenance  problems  more 
effectively.  This  is,  in  fact,  the  goal  of  the  Basic  Skills  and 
Knowledge  (BKS)  training.  Phase  I  of  the  63W10  Program  at  Aber¬ 
deen.  But,  nothing  the  ARI  researchers  have  observed  would 
suggest  that  Phase  I  training  has  a  marked  effect  on  task  perfor¬ 
mance  or  retention  (Ramsay  et  al.,  1988). 

(b)  The  "Users  Manual"  provides  clues  for  solving  the 
"mental  requirements"  problem.  It  describes  simple  mental 
processing  as  "making  gross  comparisons  ...  or  performing  simple 
computations"  (op.  cit.  p.  29) .  "Complex  mental  processes 
require  the  soldier  to  make  a  choice  . . .  based  on  subtle  but 
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discrete  clues  (e.  g. ,  ...  identifying  different  types  of 
aircraft  or  vehicles)"  (op.  cit.  p.  29).  Thus  the  manual  implies 
that  the  students  must  be  taught  how  to  discriminate,  generalize, 
and  reason  about  situations  so  well  that  the  behaviors  required 
become  as  natural  as  gross  comparisons  or  simple  computations. 

(2)  Solutions: 

(a)  Front-end  Analysis  (FEA) .  Apply  the  basic  principles  of 
FEA  described  in  the  Harless  method  (Harless,  1982) .  In  par¬ 
ticular  identify  where  soldiers  must  discriminate  or  generalize 
to  perform  effectively.  The  Army  routinely  ignores  this 
prescription  and  describes  tasks  as  lists  of  chained  procedures, 
an  inaccurate  and  misleading  approach.  We  can't  begin  to  solve 
the  skill  decay  problem  imposed  by  "mental  requirements"  unless 
we  define  those  requirements  accurately  and  operationally. 

(b)  Apply  special  techniques  and  pay  special  attention  to 
training  discriminations  and  generalizations,  as  explained  in 
Harless  (1982).  These  techniques  included  isolating  these  parts 
of  a  task,  using  verbal  mediators  or  shaping  techniques  for 
difficult  discriminations,  and  providing  added  practice.  For 
instance  it  might  be  possible  to  train  soldiers  to  use  learn 
auditory  diagnostic  cues  to  the  level  that  these  cues  would  be  as 
natural  as  gross  comparisons.  Among  the  target  cues  might  be 
universal  joint,  differential,  and  brake  noises.  We  suspect  that 
auditory  cue  training  might  be  useful  in  adjusting  the  fuel 
injector  assembly. 

(3)  Example  of  Pay-off.  For  the  "replace  fuel  injector 
pump"  task  improved  methods  for  coping  with  "mental  requirements" 
could  move  the  50%  failure  rate  point  from  2. 5-8.0  to  5.0-16.0 
weeks . 


c.  Number  of  Facts  to  Memorize/Difficulty  to  Remember 

(1)  Problem: 

(a)  Number  of  facts  contributes  substantially  to  skill  decay 
for  the  three  tasks  involving  engine  starting  problems. 
Difficulty  of  remembering  facts  is  an  added  problem  for  a 
cranking  engine  that  won't  start.  Number  of  facts  is  not  a  major 
problem  for  the  remaining  tasks,  although  the  few  facts  connected 
with  replacing  the  fuel  injector  pump  were  considered  a  bit 
difficult  to  recall  by  one  group  of  SMEs. 

(2)  Solutions: 

(a)  Good  job  aids  can  do  much  to  off-set  this  problem.  The 
key  however  is  to  make  the  facts  easy  to  get  at.  The  SMEs 
stressed  that  facts  buried  in  multiple  TMs  are  not  easy  to  find. 


A  small  well  indexed  card  catalog  with  the  most  frequently  used 
tolerances,  settings,  and  diagnostic  readings  may  be  one  answer. 
The  Skill  Retention  Model  can  help  identify  what  set  of  facts 
should  go  in  the  catalog. 

(b)  Training  solutions. 

(i)  If  a  task  has  8  or  more  facts,  pull  them  out  in  a 
condensed  list  and  give  the  trainee  practice  in  reviewing  the 
list.  Teach  three  or  four  facts  at  a  time.  Do  this  before  the 
soldiers  practice  the  whole  task,  and  then  again  after  they 
finish  the  task.  If  the  same  facts  are  used  in  other  tasks, 
prepare  a  list  of  those  tasks  so  that  the  trainees  can  see  how 
the  facts  generalize.  Show  clearly  how  the  same  fact  (e.g 
tolerance,  spec,  formula)  applies  to  other  tasks. 

(ii)  Use  memory  aids  (i.e.  mnemonics).  This  requires 
cleverness  on  the  part  of  the  instructor  to  generate  mnemonics 
which  make  sense  and  are  easy  to  remember  for  these  particular 
students  and  topics.  e.g.  the  instructor  may  need  different 
mnemonics  for  different  types  of  students. 

(3)  Example  of  Pay-off.  Successful  efforts  to  reduce  the 
skill  decay  effects  of  large  numbers  of  facts  would  move  the  50% 
failure  rate  point  for  diagnosing  a  stalling  engine  from  6.5  to 
9.5  weeks. 


d.  Motor  Control  Requirements 

(1)  Problem.  The  "User's  Manual"  and  its  first  author  state 
that  tasks  requiring  a  considerable  amount  of  motor  control,  such 
as  that  required  for  touch  typing  or  driving  a  manual 
transmission  car,  are  retained  better  than  other  tasks  of  greater 
or  lesser  difficulty.  Some  of  the  tasks,  "hard-starts",  injector 
replacement",  and  "power  loss"  require  considerable  amounts  of 
motor  control,  the  optimal  level  for  task  retention.  Other 
tasks,  such  as  "remove  and  replace  clutch",  "engine  cranks/no 
start",  and  "engine  stalls",  do  not. 

(2)  Solution.  Consider  increasing  the  complexity  of  the 
physical  performance  demands  of  the  task  during  training.  For 
example,  require  use  of  torque  wrenches  and  other  precision  tools 
even  though  they  may  not  be  required  or  used  on  the  job. 

(3)  Example  of  Pay-off.  For  the  "engine  stalls"  problem, 
the  50%  failure  point  would  move  from  6.5  to  10  weeks. 


4.3  Implications  of  the  Model  and  the  study  for  cost-training 
effectiveness  analysis  (CTEA) . 
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a.  The  examples  of  pay-off  suggest  that  the  Skill  Retention 
model  can  be  a  very  useful  tool  in  doing  CTEAs,  i.e.  in  answering 
the  question  "how  much  is  a  pound  of  training  worth".  For 
example,  we  can,  with  relative  ease  (e.g.  see  Adams,  1986), 
estimate  costs  for  any  proposed  training  solution  or  solutions 
derived  from  the  model  and  then  estimate  from  the  model,  the 
number  of  weeks  of  retention  provided  by  the  solution. 

b.  For  example,  the  model  suggests  that  a  50%  pass  rate  for 
replacing  the  fuel  injector  pump  can  be  extended  from  2.5  to  5 
weeks  (worst  rating)  if  we  provide  job  aids  that  help  the 
mechanic  move  through  multiple  TMs.  We  can  estimate  how  much  it 
will  cost  to  develop  and  implement  the  job  aid  or  even 
alternative  job  aids.  Putting  a  value  on  the  extra  2.5  weeks  is 
not  quite  as  straight  forward,  but  with  some  imagination  it  can 
be  done.  Look  at  the  cost  of  retraining,  especially  for  reserve 
units  that  meet  once  a  month.  You  can  then  make  statements  of 
the  form  "a  dollar  for  improved  training  will  save  two  dollars  in 
retraining" . 
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5.0  CONCLUSIONS 


5.1  We  conclude  that  the  Skill  Retention  Model  can: 

a.  be  administered  easily  and  at  modest  cost  to  APG  SMEs, 

b.  predict  skill  decay  for  wheel  vehicle  maintenance  tasks, 

c.  identify  tasks  that  will  be  forgotten  most  quickly, 

d.  identify  task  features  that  promote  skill  retention, 

e.  help  define  training  to  reduce  maintenance  skill  decay, 

and 

f.  support  cost  training  effectiveness  analysis. 

5.2  The  Skill  Retention  Model  is  sensitive  to  differences  in 
how  fast  63W  skills  decay  and  can  be  used  with  63W  SMEs. 
Furthermore,  prior  research  has  proven  that  the  model  is  reliable 
and  valid.  Therefore,  we  recommend  that 

a.  it  be  used  to  rate  the  remaining  63W  tasks,  and 

b.  that  its  use  in  CTEA  be  examined. 
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6.0  SUMMARY 


6.1  The  questionnaire  provided  in  the  "User's  Manual  for  Predict¬ 
ing  Military  Task  Retention"  was  completed  by  seven  wheeled 
vehicle  SMEs  at  Edgewood  Arsenal,  working  in  two  groups.  One 
group  of  SMEs  rated  five  tasks  that  had  been  selected  by  school 
personnel  as  important.  The  other  group  rated  five  tasks  that 
occur  frequently  in  the  field.  One  of  the  tasks  was  rated  by  both 
groups  since  it  had  been  selected  as  important  by  the  school  and 
occurred  frequently  in  the  field.  The  raters  were  able  to  use  the 
instrument  to  rate  ten  task  characteristics.  The  instrument 
discriminated  between  the  tasks  and  predicted  large  differences  in 
the  retention  of  different  tasks.  The  results  of  the  ratings 
suggested  that  63Ws  may  need  much  more  frequent  refresher  training 
on  some  tasks  than  might  be  expected.  The  descriptions  of  the 
rating  scales  provided  in  the  "User's  Manual  for  Predicting 
Military  Tack  Retention"  suggested  approaches  to  revising  the 
training  that  could  increase  retention. 

6.2  We  concluded  that  the  Skill  Retention  Model  is  sensitive  to 
differences  among  63W  tasks,  that  it  can  be  used  by  63W  SMEs,  and 
that  it  can  help  identify  job  aiding  and  training  procedures  to 
reduce  skill  decay.  We  also  concluded  that  it  could  be  a  useful 
tool  for  cost-training  effective  analysis. 
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7 . 0  END  NOTES 


1.  The  steering  gear  replacement  is  not  an  exception.  One 
rater  stated  that  they  exist  but  are  not  available  on  the  shop 
floor.  The  reason  for  including  this  rater  is  explained  in  the 
text. 

2.  The  reasons  for  two  sets  of  results  for  some  tasks  were 
explained  in  the  procedure  section. 

3.  For  instance,  the  initial  analysis  suggested  that  the 
diagnostic  tasks  should  be  critical.  The  field  research  failed 
to  observe  them  and  determined  that  the  remove  and  replace  tasks 
were  critical  in  terms  of  freguency  of  actual  performance. 
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APPENDIX  A:  Task  Descriptions 

Diagnose  hard  starting 
Diagnose  loss  of  engine  power 
Diagnose  stalling  engine 

Diagnose  engine  which  cranks  but  doesn't  start 
Replace  fuel  injector  pump 

Remove  and  replace  the  steering  gear  on  an  M35A2 
Remove  and  replace  the  axle  on  an  M813A1 
Remove  and  replace  the  clutch  on  an  M35A2 
Remove  and  replace  the  transmission  on  a  M35A2. 
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APPENDIX  B:  Definitions  of  Scales  in  Retention  Model 


IX.  Instructions  for  the  Paper-and-Pencil  Version 

The  task  rating  method  has  been  developed  in  both  a 
paper-and-pencil  version  (described  in  this  manual)  and  a 
computer-based  version.  This  manual  is  intended  to  serve  as 
a  basic  guide  for  those  who  are  asked  to  rate  tasks  to 
determine  their  level  of  retention.  While  it  contains  the 
essential  information  needed  to  use  the  rating  method 
properly,  situations  may  arise  for  vhich  adequate  guidance 
is  not  provided.  Further  assistance  may  be  obtained  from 
the  U.S.  Army  Training  Board  at  Comm  (804)  878-4658  or  AV 
927-4658. 

-The  Rating  Procedure 

The  task  rating  procedure  contains  ten  questions  with 
each  having  from  two  to  four  choices  of  answers.  In 
addition,  there  is  a  Definitions  section  designed  to  clarify 
the  meaning  of  each  question  and  help  with  the  selection  of 
the  most  appropriate  answer.  Xt  is  important  to  read  all  of 
the  Definitions  information  before  selecting  an  answer. 

The  paper-and-pencil  version  contains  a  Task  Retention 
Rating  Form  on  which  the  answers  to  each  question  for  each 
task  are  to  be  recorded.  A  copy  of  the  Rating  Form  is 
included  on  Page  40  of  this  manual. 


Here  i s  the  step-by-step  procedure  to  follow: 


1.  List  the  number  designation  and  title  of  each  of 
the  tasks  you  are  going  to  rate  in  the  first  two 
columns  of  the  Rating  Form.  Use  one  line  on  the 
form  for  each  task.  Tou  may  shorten  or  abbre¬ 
viate  the  task  title.  Enter  the  Military 
Occupational  Specialty  (MOS)  or  Specialty  Code 
(SC)  designation  at  the  top  of  the  form. 

If  you  are  rating  tasks  in  more  than  one 
MO S/SC,  use  separate  Ratings  Forms  for 
each. 

2.  Refer  to  the  current  task  summary 

to  obtain  a  description  of  each  task 
you  intend  to  rate.  You  may  also 
use  information  found  in  referenced  documents 
for  each  task. 

3.  Read  the  first  question  and  the  associated 
Definitions  section.  Review  the  supporting 
documentation  for  each  task  as  needed  to 
arrive  at  an  answer.  Note  the  Scale  Value 
for  the  answer  you  selected.  Write  that 
Scale  Value  in  the  box  in  the  Rating  Form 
corresponding  to  that  task  and  question. 

4.  Continue  in  this  manner  until  you  have 
answered  all  10  of  the  questions  for  Task  1 
and  have  entered  the  appropriate  Scale 
Values  on  the  Rating  Form.  Depending  on 
your  answers  you  may  be  asked  to  skip 
certain  questions.  Follow  the  instructions 
given  for  each  question. 

5.  Add  the  individual  Scale  Values  for  Task 
1  and  enter  the  total  in  the  "Total 
Score"  column  of  the  Rating  Form. 

6.  Follow  the  same  procedure  for  the  remaining 
tasks  you  wish  to  rate. 

7.  Review  the  ratings  given  to  each  task  by  other 
raters.  Resolve  differences  and  document  any 
changes  in  or  assumptions  made  about  a  task. 
When  the  scale  values  for  each  question  and 
the  final  score  have  been  agreed  upon  by  the 
raters,  record  them  on  a  separate  Rating  Form 
noting  that  it  is  the  "Approved  Task  Rating 
Form."  Supplementary  documentation  should  be 
attached  to  this  Rati.-.'  Form  to  support  the 
ratings  given. 


In  order  to  convert  the  final  ratings  to  retention 
performance  predictions,  follow  the  instructions  starting  on 
Page  38. 

The  flow  chart  on  the  next  page  shows  the  decision 
algorithm  that  is  built  into  the  series  of  10  questions.  As 
you  go  through  the  questions,  you  will  note  that  the 
instructions  contain  specific  guidance  with  respect  to  the 
decision  points  shown  in  the  chart.  It  is  important  that 
these  instructions  be  followed  carefully. 


Question  1.  Are  job  or  memory  aids  used  by  the  soldier  in 
performing  (and  in  the  performance  evaluation  of)  this  task? 


Answer  Choice 


Scale  Value 


e  Tes  1 

e  No  0 


Definitions 


Job  and  memory  aids  are  designed  to  guide  or  facilitate 
the  soldier  in  on-the-job  performance  and  to  minimize  the 
need  for  recall.  Examples  include: 


a  Memory  joggers  learned  in  school ,  such  as 
S-A-L-D-T-E 

e  Technical  manuals  or  pamphlets 

when  used  on  the  job  to  help  do  the  task  properly. 

e  Labels  or  instructions  that  are  printed  on  or 
attached  to  equipment  or  containers. 

e  Checklists#  flowcharts,  worksheets,  decision 
tables,  and  system-fault  tables. 

e  Manuals  published  by  manufacturers  to  be  used  while 
performing  maintenance  tasks  on  equipment. 

e  Instructions  on  forms  that  tell  how  to  complete 
them.  The  form  itself  is  not  a  job  aid,  even 
though  it  has  headings  that  indicate  what  should 
be  entered  in  the  form.  It  is  considered  to  be 

an  integral  part  of  the  task  itself. 

The  key  to  accurately  answering  this  question  lies  in 
the  way  the  task  is  intended  to  be  evaluated  and  performed. 


"o 


For  example,  it  is  intended  that  a  Technical  Manual  be  used 
while  performing  most  maintenance  tasks.  That  is  the  way 
these  tasks  are  taught  and  the  way  they  are  evaluated.  If  a 
job  or  memory  aid  is  not  used  while  performing  a  task,  and 
none  is  used  in  evaluating  that  task,  then  the  answer  to 
this  question  would  be  Vo. 

All  reference-dependent  tasks;  that  is,  tasks  for  which 
required  references  are  noted  in  the  conditions  section  of 
the  task  summary;  should  be  considered  job  aided. 

Tools  or  equipment  needed  to  perform  a  task  are  not  job 
aids.  However,  if  a  supervisor  is  permitted  to  aid  a 
soldier  during  the  task  performance  evaluation,  the 
supervisor  should  be  considered  as  a  job  aid. 

Write  the  Scale  Value  ("1"  or  "0")  for  the  answer  you 
select  in  the  first  column,  labeled  "Job/Memory  Aid,"  of  the 
Rating  Form. 

If  you  selected  a  Vo  answer  ("0")  to  this  question, 
skip  the  next  question  and  go  to  Question  3. 

If  you  selected  a  Tes  answer  ("1”)  to  this  question, 
answer  Question  2  on  the  next  page.  Tou  will  also  be 
reminded  of  the  fact  that  a  job  or  memory  aid  is  used  to 
perform  this  task  on  several  of  the  other  questions.  Their 
rating  will  depend  on  how  much  support  is  provided  by  the 
aid,  which  is  covered  by  the  next  question. 


~  1 


MOTE:  IP  THERE  IS  NO  JOB  OR  MEMORY  XI D, 

00  NOT  ANSWER  THIS  QUESTION. 

GO  TO  QUESTION  3 


Question  2.  How  would  you  cate  the  quality  of  the  job  or 
memory  aid? 


Answer  Choice  Scale  Value 

e  Excellent.  Using  the  job/memory  aid,  56 

a  typical  soldier  can  do  the  entire  task 
correctly  with  no  additional  information 
or  help. 

e  Very  Good.  With  the  job/memory  aid,  a  25 

typical  soldier  would  need  only  a  little 
additional  information  to  complete  the 
task. 

e  Marginally  Good.  Even  with  the  job/  2 

memory  aid,  a  typical  soldier  would  need 
important  additional  information  to 
complete  the  task. 

e  Poor.  Even  with  the  job/memory  aid,  1 

a  typical  soldier  would  need  a  great 
deal  of  additional  information  in 
ordez  to  complete  the  task. 


Definitions 


This  question  requires  you  to  think  about  the  ability 
of  the  job  or  memory  aid  to  actually  lead  the  soldier 
through  the  entire  task  without  error. 


There  are  several  dimensions  that  help  to  define  the 
quality  of  a  job  or  memory  aid. 


e  Clarity  -  an  excellent  job  aid  presents  the 

information  a  soldier  needs  to  perform  the  task  in 


a  way  that  the  soldier  can  understand;  that  is, 
the  language  and  terminology  matches  the  soldiers' 
level  of  understanding  and  reading  ability.  If 
necessary,  pictures,  diagrams,  tables  and  charts 
are  used  to  present  critical  information. 

•  Completeness  -  an  excellent  job  aid  provides  all 

of  the  information  the  soldier  needs  to  do  the  task. 
A  complete  job  aid  tells  the  soldier  what, 
when,  and  how  to  perform  at  a  necessary  level 
of  detail.  However,  an  aid  that  covers  only  a 
portion  of  a  task  very  well  is  still  less  than 
excellent  if  other  portions  are  left  uncovered  or 
are  covered  poorly. 

e  Dseability  -  an  excellent  job  aid  is  useable 
while  the  job  is  actually  being  performed.  For 
example,  a  detailed,  well-written  technical  manual 
is  still  less  than  an  excellent  job  aid  if  the 
soldiers  cannot  take  the  time  to  read  it 
or  if  it  is  physically  impossible 
to  use  the  aid  while  performing  the  task.  In  the 
same  way,  a  job  aid  may  be  excellent  under  some 
conditions  but  poor  under  others  (e.g.,  darkness). 


In  some  cases  a  task  may  have  more  than  one  job  aid. 
The  rater  should  consider  the  overall  excellence  of  the  job 
aids  in  relation  to  the  entire  task.  Only  if  the  job  aids, 
taken  together,  provide  clear,  complete  and  useful  coverage 
of  the  whole  task  can  the  rating  on  this  question  be 
excellent. 


The  following  examples  for  the  task  "Turn  On  Electrical 
Test  Panel"  may  help  in  making  your  choice: 


excellent  job  aid  -  Easy-to-read  instructions  printed 
clearly  on  the  electrical  test  panel  itself,  telling 
you  when  to  do  the  task,  what  to  do,  how  to  do  it,  and 
in  what  order  to  do  it.  Pictures  are  used  to  help 
locate  things. 

Very  Good  job  aid  -  A  booklet  that  tells  you  basically 
the  same  information  but  it  does  not  show  where  the 
knobs  and  awitches  are  located.  Operator  errors  are 
more  likely. 

75 


Marginally  Good  job  aid  -  Printed  technical 
instructions  that  contain  other  information  about  the 
test  system  mixed  in  with  the  needed  information.  Mo 
pictures  or  diagrams. 

Poor  job  aid  -  Technical  reference  manuals  in  which 
general  principles  of  operation  are  given  using  complex 
language  -  you  must  try  to  determine  the  actual 
procedure  for  yourself. 

Choose  your  answer  using  the  above  guidance  and  enter 
the  Scale  Value  for  that  answer  on  the  Rating  Form  under  the 
column  labeled  2,  "Job/Memory  Aid  Quality.” 

If  you  select  excellent  as  the  answer  to  this  question 
you  will  skip  the  next  3  questions.  Look  at  these  3 
questions  now  to  see  if  your  assessment  of  the  job  aid  is 
accurate.  In  effect,  if  a  job  aid  is  excellent  it  is  the 
same  as  saying  the  task  has  only  1  step  (read  job  aid) ,  no 
particular  sequence  to  remember  (job  aid  tells  you)  and  has 
lots  of  built  in  feedback  (job  aid  tells  you  if  you  are 
doing  each  step  correctly) .  If  you  have  any  doubts  about 
whether  the  job  aid  that  you  have  rated  excellent  meets 
these  standards,  you  should  down-grade  your  rating  to  very 
good  and  answer  the  next  3  questions. 


"4 


NOTE:  IF  TOO  RATED  THE  JOB  AID  AS  "EXCELLENT" 

DO  NOT  ANSWER  THIS  QUESTION  OR  THE  NEXT  TWO  QUESTIONS. 

GO  TO  QUESTION  6 

Question  3.  Into  how  many  steps  has  the  task  been  divided? 


Answer  Choice  Scale  Value 

s  One  step  25 

e  Two  to  five  steps  14 

e  Six  to  ten  steps  12 

e  More  than  ten  steps  0 


Definitions 

For  purposes  of  this  rating  you  should  use  the  number 
of  evaluated  performance  measures  listed  in  the  task  summary 
under  the  Evaluation  Guide  as  the  number  of  steps.  Where 
there  are  non-evaluated  sub-steps,  these  should  also  be 
counted.  However,  do  not  include  performance  measures 
relating  to  whether  the  soldier  performed  the  task  steps  in 
sequence  or  within  a  certain  time  period.  These  are  viewed 
only  as  "scoring  steps"  and  not  as  "task  steps"  for  this 
question. 

If  the  reference  material  does  not  provide  sufficient 
information,  or  if  you  feel  that  a  task  has  not  been 


accurately  divided  into  performance  steps,  the  following 
guidance  may  be  helpful: 


•  A  step  is  a  separate  physical  or  mental  activity 
within  a  task,  which  has  a  well-defined,  observable 
beginning  and  ending  point.  It  must  be  performed 
to  complete  a  task  correctly.  Thus,  "Identifying  a 
Tank"  is  one  step,  even  though  a  number  of  mental 
operations  are  needed  to  arrive  at  the  correct 
answer  (e.g.,  note  location  of  turret,  count 
number  of  road  wheels,  etc.)*  These  operations, 
however,  are  not  observable  and  are  not  scored  as 
separate  steps. 

•  Steps  should  include  all  safety-related 
activities,  even  though  they  are  not  directly 
part  of  the  task.  The  step  "Check  Backblast  Area" 
is  an  observable  step  and  is  graded  when  measuring 
proficiency  on  preparing  the  LAW  for  firing. 

a  Tasks  involving  assembling  or  disassembling  a 
piece  of  equipment  tend  to  be  multi-step  tasks. 
"Assembling  the  M16  Rifle"  would  be  an  example 
of  a  multi-step  task  and  it  is  scored  as  such. 

a  A  step  in  a  higher  Skill  Level  task  (E-5  or  above) 
may  be  a  separate  task  at  a  lower  skill  level. 

This  is  as  it  should  be,  since  one  must  assume 
that  the  procedure  was  learned  earlier  and  is 
no  longer  a  separate  task.  For  example, 

"Identify  Terrain  Features"  is  assumed  at 
higher  skill  levels  and  is  simply  one  step 
in  a  task  such  as  "Navigate  with  a  Map."  At 
Skill  Level  One  it  is  a  separate  task  with  many 
steps  of  its  own. 


i 
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Note  that  this  question  is  skipped  if  the  job  aid  for 
this  task  was  judged  to  be  excellent.  This  is  consistent 
with  the  definition  of  an  excellent  job  aid  as  one  that 
provides  the  soldier  with  complete  information  about  each  of 
the  steps  of  the  task.  Obviously,  the  number  of  steps  in  a 
task  would  be  Irrelevant  to  the  soldier  who  had  forgotten 
them  if  the  job  aid  presented  them  clearly  and  completely. 
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If  steps  are  repeated  in  a  task,  the  instruction  to 
repeat  them  should  be  counted  as  a  step,  but  the  repeated 
steps  themselves  should  not  be  recounted.  For  example,  when 
shifting  indirect  fire  the  soldier  would  repeat  the  steps  of 
estimating  range,  and  communicating  information  to  the  gun 
crew  several  times  until  the  target  was  destroyed.  Each 
repetition  would  not  constitute  a  separate  step. 

If  it  is  agreed  that  there  are  clearly  more  than  10 
steps  to  a  task,  there  is  no  point  in  trying  to  resolve 
differences  of  opinion  about  the  actual  number.  Anything 
over  10  is  given  a  0  Scale  Value  regardless  of  the  actual 
number . 

Determine  your  answer.  Enter  the  Scale  Value  for  this 
answer  on  the  Rating  Form  in  the  third  column  labeled 
"Number  of  Steps." 

If  you  select  the  first  answer  (One  step) ,  skip  to 


Question  6 


MOTE:  ir  THE  TASK  HAS  ONLY  ONE  STEP, 
GO  TO  QUESTION  6 


Question  4.  Aze  the  steps  in  the  task  required  to  be 
performed  in  a  definite  sequence? 


Answer  Choice 
•  None  are 

e  All  are 

e  Some  are  and  some  are  not 


Scale  Value 

10 

5 

0 


Definitions 


Some  tasks  are  composed  of  steps  that  can  be  performed 
in  any  sequence.  For  example,  "Identify  Terrain  Features  on 
a  Map"  is  a  task  that  is  not  scored  for  sequence  in  the 
evaluation  guide  of  the  task  summary.  Such  tasks  should  be 
given  a  Scale  Value  of  "10". 

Other  tasks,  such  as  "Splint  a  Fracture,"  are  made  up 
of  steps  that  have  only  one  correct  sequence*  Failure  to 
follow  the  particular  sequence  results  in  a  "NO  GO"  on  that 
task.  These  tasks  should  be  given  a  Scale  Value  of  "5". 

A  task  that  is  a  mixture  of  sequenced  and  non-sequenced 
steps  should  be  given  a  Scale  Value  of  "0".  "Perform 
Operator  Maintenance  on  an  M16A1  Rifle”  is  such  a  task. 

Only  steps  1-5  are  scored  for  sequence. 


The  reasoning  behind  this  question  is  that  it  is  easier 
to  remember  to  do  a  task  when  sequence  does  not  matter. 
However,  if  sequence  is  scored,  it  is  easier  to  remember  a 
specific  sequence  for  all  steps  than  for  only  some  of  the 
steps. 

If  a  task,  or  parts  of  it,  are  supposed  to  be  performed 
in  sequence,  there  must  be  a  statement  to  that  effect  in  the 
task  summary;  e.g.,  "Do,  in  order,  all  steps  to  clear  the 
object  from  the  casualty's  throat."  In  the  absence  of  any 
statement  about  sequence,  it  should  be  assumed  that  sequence 
for  that  task  is  not  scored,  even  though  there  may  be  a 
natural  or  preferred  order  to  doing  the  steps. 

Choose  your  answer.  Enter  the  Scale  Value  for  your 
answer  in  the  fourth  column  (labeled  "Sequence")  of  the 
Rating  Form. 


MOTE:  IF  THE  TASK  HAS  ONLY  I  STEf 

SKIP  THIS  QUESTION  AND  GO  TO 
QUESTION  € 

Question  5.  Does  the  task  provide  built-in  feedback  so  that 
you  can  tell  if  you  are  doing  each  step  correctly? 

Answer  Choice  Scale  Value 

e  Has  built-in  feedback  for  all  steps  22 

•  Has  built-in  feedback  for  most  steps  19 

(501  and  above) 

a  Has  built-in  feedback  for  only  a  few  steps  11 
(up  to  50%) 

a  Has  no  built-in  feedback  0 

Definitions 


Examples  of  tasks  that  provide  built-in  feedback  are: 


a  Disassembling  a  piece  of  equipment  in  which 

removing  one  section  automatically  uncovers  the 
next  section  (e.g.,  opening  up  a  container  to 
remove  contents) . 

a  Equipment  operation  in  which  the  steps  form  a 
logical  or  natural  progression.  For  example, 
radio  operators  are  expected  to  adjust  or  turn  on 
several  dozen  switches  in  a  certain  order 
when  "powering  up"  their  radio  sets. 

However,  the  equipment  is  arranged  so  that 
the  operator  can  follow  a  natural  right-to-left 
or  lef t-tc-right  progression. 


so 


•  Assembling  a  subpart  that  does  not  fit  the  larger 
assembly,  thus  indicating  that  some  earlier  step 
was  incorrect. 

•  Any  task  where  there  is  some  observable  effect 
due  to  the  soldiers'  actions,  e.g.,  warning  light, 
buzzer,  meter  reading,  and  the  like. 

For  some  tasks,  the  completion  of  the  task  provides  an 
automatic  check  on  the  correctness  of  it.  For  example, 
'"Changing  a  Tire”  wouli  have  some  of  those  characteristics 
(e.g.,  parts  left  over,  wheel  does  not  turn).  The  impact  of 
an  artillery  round  would  have  a  similar  effect.  However, 
such  end-of-task  feedback  may  not  assist  the  soldier  in 
performing  the  steps  of  the  task  correctly  in  the  first 
place,  and  should  not  be  considered  in  answering  this 
question. 

The  important  point  to  consider  in  selecting  an  answer 
to  this  question  is  whether  the  feedback  to  the  soldier 
indicates  the  correctness  of  his  performance  at  each  step. 
Feedback  that  simply  indicates  that  the  step  was  completed 
is  not  the  kind  of  feedback  that  this  question  is 
addressing. 

Steps  that  have  the  least  built-in  feedback  tend  to 
have  many  branching  routines  ("If  A,  then  B”),  or  have 
safety  checks  that  break  the  flow  of  a  task's  steps  (e.g., 
"Place  the  selector  on  SAFE  before  cleaning  the  rifle"). 

Do  not  confuse  the  feedback  that  a  performance  test 
administrator  may  give  to  a  soldier  to  allow  him  to  continue 


the  test  after  an  error  is  made,  with  the  feedback  that  is 
being  addressed  by  this  question.  We  are  concerned  only 
with  feedback  that  is  an  inherent  part  of  the  task,  not  an 
artifact  introduced  by  test  conditions. 

Before  answering  this  question  you  may  wish  to  look 
back  to  question  3  to  see  how  many  steps  you  identified  for 
the  task.  You  need  to  consider  each  of  them  in  arriving  at 
the  correct  answer  to  this  question. 

Enter  the  Scale  Value  for  your  answer  on  the  Rating 
Form  in  the  column  headed  "Feedback.*1 


Question  6.  Does  the  task  or  part  of  the  task  have  a  tine 
Unit  for  its  conpletion? 

Answer  Choice  Scale  Value 

o  There  is  no  tine  linit  40 

•  There  is  a  tine  linit,  but  it  is  fairly  35 

easy  to  neet  under  test  conditions 

•  There  is  a  tine  linit  and  it  is  difficult  0 

to  neet  under  test  conditions 

Definitions 

The  first  choice  neans  that  no  tine  linit  has  been 
established  for  the  task  or  any  part  of  the  task,  so  that  a 
"GO"  nay  be  achieved  even  though  one  soldier  nay  take  nuch 
longer  to  do  the  task  than  another  soldier.  This  choice  is 

also  appropriate  when  a  tine  linit  is  so  liberal  that  no  one 

ever  fails  to  neet  it. 

The  second  choice  above  applies  to  those  tasks,  such  as 
"Assenble  the  M60  Machinegun,"  that  have  a  tine  linit  that 
sone  soldiers  nay  find  difficult  to  neet.  In  this  case,  the 
task  sunnary  has  set  a  tine  linit  that  "pressures  the 
average  soldier  a  bit,  but  only  a  few  would  get  a  "MO  GO" 
because  of  it. 


The  third  choice  is  for  tasks  that  have  a  time  limit 
that  is  difficult  to  meet.  Safety  and  combat-related  tasks, 
such  as  "Sight  a  Target  Through  the  Gunner's  Telescope" 
vithin  10  seconds  would  fall  into  this  category.  Soldiers 
being  tested  on  this  kind  of  task  often  get  a  "MO  GO"  on  the 
basis  of  time  alone. 

Time  limits,  if  any,  are  indicated  in  the  task  summary 
in  the  standards  section  or  as  the  last  item  in  the  list  of 
performance  steps  to  which  they  apply.  Some  examples  are: 
"Put  On,  Clear  and  Check  Mask"  within  9  seconds,  or 
"Complete  Steps  1  Through  5  in  9  Seconds  or  Less." 

(Remember,  however,  that  the  time  limit  statement  itself  is 
not  counted  as  a  step.)  If  no  time  limit  statement  is  found 
in  the  task  summary  it  may  be  assumed  that  there  is  none. 


Question  7.  How  difficult  are  the  mental  processing 
requirements  of  this  task? 

Answer  Choice  Scale  Value 

•  Almost  no  mental  processing  requirements  37 

•  Simple  mental  processing  requirements  28 

e  Complex  mental  processing  requirements  3 

e  Very  complex  mental  processing  requirements  0 

Definitions 

This  question  usually  cannot  be  answered  entirely  on 
the  basis  of  the  task  summary  (as  could  the  time  and 
sequence  questions)  but  often  must  be  deduced  from  a  careful 
reading  of  the  summary  and  first-hand  knowledge  of  the  task 
itself.  This  question  gets  at  the  difficulty  of  the  thought 
processes  that  a  soldier  must  carry  out  during  task 
performance.  Such  processes  as  often  described  by  terms 
such  as  •thinking,"  "reasoning,"  "analyxing,"  "judging," 
"inferring",  and  "problem  solving". 

Be  careful  not  to  confuse  this  question  with  the  next  2 
questions  which  deal  with  the  number  of  facts,  terms,  etc., 
that  must  be  memorised  and  the  difficulty  of  those  facts, 
terms,  etc.,  to  memorise.  Here  we  are  concerned  only  with 

8  5 


mbat  yon  have  to  do  with  the  recalled  information  in  order 
to  perform  the  task  correctly. 

A  task  requires  almost  mo  mental  processing  if  it  is 
essentially  physical*  or  highly  repetitive  (e.g.*  "Marching 
in  tine, "  "Saluting"). 

A  task  requires  simple  mental  processing  if  it  involves 
making  gross  comparisons;  estimating  relative  size*  weight* 
or  distance;  or  performing  simple  computations. 

Complex  mental  processes  require  the  soldier  to  make  a 
choice  or  decision  based  on  subtle  but  discrete  clues  (e.g.* 
prioritizing  fixed  targets*  identifying  different  types  of 
aircraft  or  vehicles) . 

A  task  requires  very  complex  mental  processes  if  it 
requires  rapid  decisions  based  on  detailed*  technical 
information*  often  under  stress  (e.g.*  planning  an  attack, 
troubleshooting  complex  equipment) . 

In  answering  this  question*  consider  the  impact  of  a 
job  or  memory  aid  on  the  thinking  requirements  of  this  task. 
However*  note  that  job  aids  are  generally  less  helpful  in 
the  area  of  higher  thought  processes  than  they  are  in  the 
areas  of  rote  memory  or  proceduralized  (step-following) 
tasks.  Nevertheless,  an  excellent  aid  may  reduce  a  very 
complex  mental  processing  task  to  a  complex  mental 
processing  task. 

so 


Question  8.  How  many  facts,  terns,  names,  rules  or  ideas 
must  a  soldier  memorize  in  order  to  do  the  task? 


Answer  Choice  Scale  Value 

•  Hone  (or  the  job/memory  aid  20 

provides  all  necessary  information) 

e  A  few  (1-3)  18 

e  Some  (4  -  8)  13 

e  Very  many  (more  than  8)  0 


Definitions 

This  question  gets  at  the  number  of  isolated  pieces  of 
information  a  soldier  must  remember  in  order  to  do  the  task 
(not  how  hard  it  is  to  remember  them,  which  is  asked  in  the 
next  question) . 

Examples  of  the  types  of  information  that  may  have  to 
be  remembered  are: 

\ 

a  Military  nomenclature  (terms) 

•  Conversion  formulas 
a  Codes  or  call  numbers 

a  Technical  names,  specifications  or  tolerances 
a  Doctrinal  principles  or  rules  of  thumb 


Remember  to  consider  the  impact  of  the  job  or  memory 
aid  (if  any)  in  answering  this  question.  If  there  are 
facts,  terms,  etc.,  that  are  needed  in  order  to  do  the  task, 
but  some  or  all  are  covered  in  the  job  aid,  your  answer 
should  reflect  this. 

The  steps  required  to  perform  the  task  should  not  be 
considered  in  answering  this  question.  It  is  the  facts, 
terms,  etc.,  that  must  be  remembered  to  do  these  steps  that 
are  being  addressed  by  this  question,  not  the  steps 
themselves. 

This  question  (and  the  next  one)  do  require  some 
judgment  about  the  level  of  experience  of  the  typical 
soldier  doing  the  task  being  rated.  One  could  safely  assume 
that  the  typical  soldier  who  has  completed  AIT  knows  basic 
military  terminology  and  concepts,  and  that  these  should  not 
be  included  in  deciding  on  the  answer  to  this  question. 

Those  facts,  terms,  etc.,  unique  to  the  task  being  rated, 
however,  should  be  included. 

Here  again,  if  the  number  of  facts,  etc.,  is  obviously 
greater  than  8,  there  is  no  need  to  seek  consensus  on  this 
question  -  it  will  get  a  Scale  Value  of  "0"  regardless  of 
the  outcome. 

Select  your  answer  and  then  enter  the  Scale  Value  for 
that  answer  in  the  column  labeled  "Number  of  Facts." 
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Question  9.  How  hard  are  the  facts,  terms,  that  must  be 
remembered? 

Answer  Choice  Scale  Valoe 

e  Hot  applicable  -  there  are  none  to  34 

remember  or  the  job  or  memory  aid 
provides  all  of  the  needed  information 

e  Not  hard  at  all  -  the  Information  is  31 

simple 

e  Somewhat  hard  -  some  of  the  information  12 

is  complex 

a  Very  hard  -  the  facts,  rules,  terms,  0 

etc.,  are  technical  or  specific  to  the 
task  and  must  be  remembered  in  exact 
detail 


Definitions 

This  question  rates  the  difficulty  of  the  facts,  terms, 
etc.,  needed  to  do  the  task  (even  if  there  are  only  a  few). 

Facts  and  terms  that  have  a  close  connection  to  the 
task  itself  are  more  likely  to  be  remembered.  For  example, 
the  terms  "firing  pin,"  and  "whip  antenna"  have  a  logical 
relationship  to  their  function  and  are  easy  to  recall. 
Specific,  detailed,  or  technical  information  that  is 
unrelated  to  the  task  is  more  difficult  to  recall.  Call 
signs  and  radio  frequencies  are  examples  of 
difficult-to-recall  information  since  they  are  purposely 
assigned  at  random  but  must  be  used  with  precision.  Also, 
unorganized  facts  and  terms  (e.g.,  much  military 


nomenclature)  are  more  likely  to  be  forgotten  than  facts  and 
terms  that  are  part  of  a  system  (e.g.,  the  phonetic 
alphabet) . 

The  amount  of  help  provided  by  job  and  memory  aids 
applies  very  directly  to  this  question.  They  are  often 
designed  specifically  to  help  the  soldier  recall  information 
that  is  quickly  forgotten  (e.g.*  6-A-L-U-T-E) . 

Choose  your  answer  and  then  enter  the  Scale  Value  for 
your  answer  in  the  column  labeled  "How  Hard  to  Remember." 


Question  10.  What  are  the  motor  control  demands  of  the 
task? 


Answer  Choice  Scale  Value 

e  Hone  2 

e  Small  but  noticeable  degree  of  0 

motor  control  required 

e  Considerable  degree  of  motor  16 

control  needed 

e  Very  large  degree  of  motor  3 

control  needed 


Definitions 

This  question  has  to  do  with  the  level  of  precision 
and/or  accuracy  of  finger ,  hand  and  arm  movements,  not  with 
large  body  movements.  Thus  a  task  would  be  given  a  nor-  if 
it  involves  only  sheer  physical  strength  or  simple, 
reflexive  actions  (e.g.,  pushing,  lifting,  carrying). 

A  email  but  noticeable  degree  of  accuracy/precision  is 
required  by  tasks  such  as  driving  a  nail  or  adjusting  a 
carburetor  screw. 

*  A  considerable  degree  of  motor  control  is  needed  for 
tasks  such  as  typing,  driving  a  manual  transmission  car  or 
tracking  a  moving  target. 

A  task  requiring  a  eery  large  degree  of  demands  would 
be  the  repair  of  a  very  delicate  piece  of  equipment,  such  as 
a  microcircuit  chip,  or  sending  Morse  code  using  a  key. 


Some  tasks  may  combine  both  a  strength  component  and  a 
motor  control  component.  For  example,  a  fairly  heavy  piece 
of  equipment  may  have  to  be  positioned  in  a  precise 
location.  In  such  cases,  a  value  of  considerable  or  even 
very  large  would  be  appropriate,  depending  on  the  degree  of 
motor  control  required. 

Almost  all  tasks  require  some  speaking  skills  end  many 
of  them  require  written  skills  as  well.  While  these  are 
indeed  complex  motor  demands,  they  are  considered  to  be 
already  in  the  repertory  of  the  typical  soldier,  and 
therefore  should  not  be  included  in  making  this  rating. 
However,  typing  or  sending  Morse  code  may  be  an  integral  and 
unique  part  of  a  task.  Therefore,  they  should  be  considered 
in  selecting  a  Scale  Value  for  this  question. 

While  the  Scale  Values  associated  with  this  question 
may  appear  to  be  incorrect,  since  they  are  not  in  decending 
order  as  are  all  the  others,  they  are,  in  fact,  the  correct 
values.  A  task  with  a  small  degree  of  motor  control  proves 
to  be  more  difficult  to  remember  than  a  task  with  a 
considerable  degree  of  motor  control.  This  finding  is 
reflected  in  the  low  value  for  the  former  ("0")  and  the  high 
value  for  the  latter  ("IS"). 

Select  your  answer  and  enter  the  Scale  Value  in  the 
column  labeled  "Motor  Control  Requirements." 
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III.  Calculating  a  Total  Score 

This  completes  the  instructions  on  the  rating  procedure 
itself.  Check  back  to  make  sure  you  answered  all  the 
questions  that  apply  to  the  task  being  rated.  The  questions 
you  skipped  (if  any)  should  be  blank  on  the  Rating  Form. 

All  others  should  have  a  0  or  a  number  written  in.  If  you 
rated  the  first  question  a  "1"  there  should  be  a  rating  for 
the  second  question;  if  you  rated  the  first  question  a  *0*, 
the  second  question  should  be  blank.  If  question  two  has  a 
rating  of  "56"  the  next  3  questions  should  be  blank. 

Remember,  if  you  change  a  rating  for  one  question  then 
you  must  also  change  all  the  other  questions  that  would  be 
effected  by  that  change  and  adjust  your  Total  Score 
accordingly.  (See  the  flow  diagram  on  page  12  for  a  graphic 
representation  of  how  the  questions  interact.) 

The  Total  Score  should  be  the  one  that  reflects  the 
input  of  all  team  members.  When  this  process  has  been 
completed,  a  new  Rating  Form  should  be  prepared  and  dated 
that  is  labeled  "Approved  Task  Rating  Form."  That  form  will 
be  the  one  kept  for  future  reference.  Subsequent  changes  in 
the  way  a  task  is  performed  as  reflected  in  the  task 
summary,  will  probably  require  that  its  rating  be  revised. 

At  that  point  a  new  form  should  be  prepared  and  dated 
reflecting  those  changes. 


IV.  Osiog  the  Performance  Prediction  Tables 

The  two  Performance  Prediction  Tables  (pp.  41  and  42)' 
provide  the  performance  predictions  for  tasks  that  have  been 
rated.  The  numbers  within  the  body  of  the  table  represent 
the  expected  proportion  of  soldiers  in  a  unit  able  to 
perform  a  task  correctly  after  up  to  1  yeai  of  no  practice 
since  a  task  was  last  performed  correctly. 

The  first  Table  gives  these  predictions  at  monthly 
intervals,  up  to  12  months.  The  second  Table  gives  these 
estimates  at  weekly  intervals,  up  to  26  weeks. 

To  find  a  specific  task  retention  prediction,  first 
locate  the  score  in  the  left  column  that  corresponds  closest 
to  the  obtained  Total  Score.  Then  read  across  either  Table' 
until  you  reach  the  time  interval  you  are  concerned  about 
(shown  at  the  top  of  the  Table) .  The  Table  entry  at  that 
point  will  be  the  proportion  of  soldiers  that  could  be 
expected  to  perform  the  task  correctly  at  that  time 
interval. 

For  example,  the  Total  Score  from  the  Rating  Form  on 
Task  X  is  "140. "  You  want  to  know  what  percentage  of 
soldiers  can  still  perform  the  task  4  months  after  they  last 
practiced  it.  The  entry  in  the  first  table  under  "4" 
(months)  is  36  percent.  The  entry  in  the  weekly  table  is 
also  36  percent. 


A  second  way  of  using  the  tables  is  as  follows:  If  a 
task  has  a  score  of  "140,”  how  often  should  sustainment 
training  be  provided  in  order  to  have  at  least  50  percent  of 
the  soldiers  proficient  on  that  task  at  all  times?  Looking 
at  the  line  on  the  table  next  to  the  value  "140,"  we  see 
that  46  percent  is  in  the  column  headed  "3"  (months).  This, 
then,  is  the  training  frequency  required  to  sustain  a  level 
of  proficiency  on  that  task  of  approximately  SO  percent. 

A  third  way  to  use  the  tables  is  to  determine  what 
level  of  proficiency  you  would  expect  in  your  unit  if  you 
could  only  provide  sustainment  training  every  "X"  months  or 
weeks.  By  looking  at  the  Total  Score  for  each  of  your  tasks 
you  can  see  what  the  percentage  is  for  each  one  under  the 
appropriate  months  or  weeks  column.  For  those  tasks  with  a 
rating  of  "130"  for  example,  the  percentage  of  proficiency 
at  a  4-month  frequency  of  training  schedule  is  25  percent; 
for  tasks  with  a  rating  of  "170"  it  is  81  percent,  and  so 


on. 


Task  Retention  Rating  Form 
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♦Performance  at  "GO"  level  of  Proficiency 


•Performance  at  "GO"  level  of  Proficiency 


APPENDIX  C:  Questionnaire  Administration  Protocol 


1.  I'm  Chavis  Patterson 

2.  I'm  here  from  ARI ,  the  Army  Research 
Institute. 

3.  TRADOC  has  asked  us  to  help  them  improve  63W 
training.  They  have  some  concern  about  how 
well  63W  are  doing  their  job. 

4.  The  approach  we're  taking  today  is  to  find  out 
if  the  63W  are  forgetting  some  important 
material  before  they  get  a  chance  to  practice 

it  in  the  field. 

We  know  the  time  it  takes  to  forget  different 
kinds  of  Army  tasks  but  we  need  SMEs  to  tell 
what  kinds  of  tasks  these  are. 

5.  That's  where  you  come  in  Mr. 


10.  ARI  developed  this  10-item  questionnaire  for 
field  commanders  so  that  they  can  find  out  how 
often  they  should  train  their  people  on  each 

of  their  unit's  tasks. 

11.  Lets  look  at  the  questionnaire  rating  form 
and  see  what  it  involves. 

12.  The  answer  sheet  has  the  10  items  right  on  it. 

13.  Here's  the  explanation  for  each  of  the 
columns . 

14.  Now  let's  look  at  the  tasks  we're  going  to 
rate.  These  task  lists  are  taken  directly 
from  the  organizational  manuals  that  the  63Ws 
have  available. 

15.  The  task  name  is  in  the  center,  and  more 
specific  info,  is  in  the  upper  right  corner. 

The  steps  for  the  task  are  listed  down  the 
left  side.  There  is  an  arrow  in  the  4th 
column  to  show  the  steps  that  are  done  in 
sequence.  The  5th  column  shows  what  the 
result  should  be,  and  the  last  two  columns 
show  the  safety  and  tools  required. 

16.  Let's  look  at  the  first  task.  Are  you  familiar 
with  this  task  or  should  we  look  at  another 

one?  Is  this  the  way  the  task  is  done  or 
should  be  done?  Any  comments? _ 
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17.  Now  let's  go  to  the  questionnaire. 


END  OF  ADMINISTRATION: 

l.We  have  gotten  your  unbiased  opinion.  Next  we 
have  to  get  you  together  with  the  other  SMEs 

to  get  a  consensus  opinion  from  all  of  you.  Can  we  meet  again 
at  ? 
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